The statements in this section merely provide background information related to the present disclosure and may or may not constitute prior art.
With rapid development of digital acquisition and processing, the dynamic video becomes one of the most popular media formats in daily life and scientific research. During video processing and playing, video quality and effect are heavily influenced by illumination. In addition, different photographing view points also lead to different appearance. As a result, to achieve observation of an object from a free view point and to generate different effects under varying lighting conditions, free viewpoint relighting needs to be performed for the object based on dynamic videos.
Presently, object relighting methods mainly comprise object relighting based on parametric illumination model and object relighting based on image.
In the relighting method based on photometric stereo, the object is captured under a plurality of controllable lighting conditions or lightings, and combined with a parametric illumination model, geometric property and reflectance peculiarities on the object surface may be recovered so that the object may be performed with relighting under a new lighting condition based on the illumination model, and the object is rendered according to a free view to be displayed. However, because scenes are collected under a plurality of controllable lightings in this method and the geometric information of the object surface is recovered based on an infinite-distance camera model during computing, the relighting method based on photometric stereo is used mainly for single view static scenes. And presently, even few relighting methods are available for dynamic scenes whereas expensive high speed cameras are needed. And because a parametric illumination model, such as a Lambertian lighting model or a spherical harmonics model, has to be resolved, the relighting method of a dynamic scene is normally limited to the processing of a Lambertian surface.
On the contrary, in the relighting method based on images, the collected object images under a plurality of dense controllable lightings are used as basis vectors, and the relighting of the object is achieved by the linear combination of the basis vectors. Because the collected images are combined as the basis vectors without resolving parameters in the resolution illumination model during the process, authenticity of the relighting effects is superior. However, the method has to be performed by sampling the object images under very dense lightings, the sampling cost and complexity thereof are high. And due to the absence of three-dimensional geometric information, free-view rendering can hardly be achieved.